ON THE CONTINUITY AND LESCHE STABILITY OF TSALLIS 
AND RENYI ENTROPIES AND Q-EXPECTATION VALUES 
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Abstract. It is shown that the Renyi and TsalUs entropies and the q-expectation 
values, are continuous and stable if g > 1 and are not continuous and instable 
for uniform finite distributions if i; < 1. 



1. Introduction 

Experimental robustness is a natural criteria of physical quantities requiring that 

A physically meaningful function of a probability distribution should 
not change drastically if the underlying distribution function is 
slightly changed. 

Lesche in 1982 has given a mathematical formulation of the above requirement, 
called stability, and proved that the entropy of Renyi is not stable p]. Based on 
Lesche's reasoning later on Abe has shown that the Tsallis entropy is stable [2]- 
Lesche stability became a criteria in distinguishing and favoring one of the many 
different entropies in non-extensive thermostatistics [3l[4l[5l[6l[7l[8l[9] and the proofs 
of Lesche and Abe become one of the arguments in favoring Tsallis entropy to Renyi. 
Lesche stability as a proper concept of experimental robustness was questioned and 
attacked by several authors [101 El [13 ■ They have collected physical arguments 
claiming that Lesche stability do not express properly the physical content of ex- 
perimental robustness. Lesche and Abe rejected these arguments [T3tll4j. Recently 
Abe recognized that the central quantities of non-extensive statistical mechanics, 
the q- averages [IS], are Lesche instable [H]. This important observation somehow 
invalidates the whole mathematical framework of non-extensive thermostatistics, 
therefore several authors argued again that Lesche stability is a too strict concept 
for physical applications and suggested different modifications \T7 \ [TS t fT9]. 

The concept of experimental robustness is a lousy continuity requirement and en- 
ables several mathematical formulations. Considering this resemblance to continu- 
ity the instability of the Renyi entropy Sr ([T]) and the stability of Tsallis entropy St 
^ is somehow paradoxical, because the Tsallis entropy S't = (1 — e*^^"*-'"^")/ (q — 1) 
(where 0<g7^1)isa smooth function of the Renyi entropy. 

In the following we investigate some mathematical concepts releted to the Renyi 
and Tsallis entropies and q-expectation values. We introduce a local form of Lesche 
stability, that, according to our opinion, expresses best the physical content of 
experimental robustness. 
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2. Continuity of functions of probability distributions 

The simplest formulation of experimental robustness is continuity. Recall the 
following notions. 

The set of infinite discrete probability distributions is 

D := {pe/i| |b||i = l, >0, i<EN}cl\ 

Here the norm is used as the natural concept of distance |20) . 

Let X be a normed space with norm || ||. 

Definition 1: A function f : D ^ X \& continuous at p if 

iVe>0){3S>0)iVr){\\r-ph<S^\\f{r)- f{p)\\<e). 

f is continuous if it is continuous at every p £ D. 

Note that if there is a positive number Cp so that ||/(?') — f{p)\\ < CpH?- — p||i 
then / is continuous at p. 

Definition 2: A function f : D ^ X is uniformly continuous if 

(Ve > 0)(3<5 > 0)(Vr,p)(||r - Pill < S ^ ||/(r) - f{p)\\ < e). 

Note that if there is a positive number c so that 11/(7") — /(p) || < c||r — p||i then 
/ is uniformly continuous. 

Continuity is a local property while uniform continuity is a global property. 
Observe that the negation of continuity reads as follows: 

(3p)(3e>0)(V5>0)(3r,||r-p||i <<5)(||/(r)-/(p)|| > e) 

and the negation of unformly continuity: 

(3e>0)(V<5>0)(3r,p,||r-p||i <<5)(||/(r)-/(p)|| > e). 

2.1. Kq. The Banach space of real sequences for which of the corresponding series 
is convergent at the power g, is denoted by Z'', and the Banach space of bounded 
sequences is denoted by l°° . We know that if fc e and 1 < g, then k £ l'^ and 
\\k\\q < ll^lli- Therefore the q-norm, as the function ||.||q : l^ ^ M.,k i-^ \\k\\q 
function, is uniformly continuous. 

Proposition 1 The function D ^ l^, p ^ p'^ := (p^)igN is uniformly continuous. 

Proof. According to the mean value theorem of differential calculus 

Ik'' -p'^lli "P^l - = iW^^Ph- 

■ 

Note that ||p«||i = \\p\\q. 
Corollary 1.1 The Renyi entropy 

(1) Sr: D^R, p^^ln||p||„ 



is continuous and the Tsallis entropy 

(2) St: D^R, pi 

if 1 < g is uniformly continuous. ■ 

The expectation value of A — (^JigN G l°° 



i-\\p\U 
g-1 



D^R, p^{A\p)^Yl ^^P^ 
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is uniformly continuous. 

In general, if $ : ^ Z? is a given function, then the ^-expectation value of A 

is 

D-^R, p^(A|$(p)). 

If $ is (uniformly) continuous, then the ^-expectation value is (uniformly) con- 
tinuous. 

g 

Corollary 1.2 The q-expectation value, where ^{p)i := \\p<i\\-i (^he quotient of 
continuous functions) is continuous. 

2.2. q<l. In this case the summability ofp'' for p £ D is not automatic. Therefore 
the previous functions (entropies and q averages) are interpreted on the set: 

D, := {p eD\ e I'}. 

Proposition 2 The function Dq ^ l^, p ^ p'^ is, not continuous at finite uniform 
distributions. 

Proof: Let be n S N a given number and 

p:= ("i,..., 1,0,0,...) eD, 

therefore the number of nonzero elements is n. In the following we will use the 
notation 



(3) 



1 



For all < ^ < 1/2 let us define 



(4) 

where 



1-S 



,0 



d_ 

m 



1 ^ I 5 



m>5^ (1 + g(5ni"«) . 
Then \\rs — p\\i — 2(5, however 

II Pa -P' 111 = ((1 - 5Y - 1)71^-9 + 5''m^-'i > 1. 



Since the logarithm and the identity are injective continuous functions, we have: 
Corollary 2.1 The Renyi and Tsallis entropies, if g < 1, are not continuous.B 
Note that the proof of the previous proposition is essentially identical that of 
Lesche in [13], regarding the instability of Renyi entropy. However, the above 
argumentation is not applicable in the case 1 < q. In particular, it is impossible to 
determine m so that ■m^~'^ > 5~'^{1 + (1 — (1 — S)'')n^~''), because then the direction 
of the inequality is reversed by the negative powers 

(59 



^9-1 < 



<(5«. 



(1 (1 - (1 - (5)9)ni-'?) 

Hence, there is no to e N that satisfies the inequality, if (5 < 1. 

Let us know investigate the continuity of the expectation values. Here it is not 
enough to show that the function p i— > is not continuous, because the strong 
convergence (convergence in norm) does not follow from the weak convergence. 
What we show is that p i— > (A|p'?/||p'^||i) is not continuous for a large set of A e 
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Let be p and p'^ are chosen as previously. Then 

(5) 
and 



Therefore 
A 



lb? Ill 



WpUi 


= (1 




V5i 


m^'i. 


pi 




61 ( 


1 


1 


Iblli 






n 





llp^lll 



51 



n ^ — ' TO . — ' 



(6) 



51 



mi-^\p1\\i 

-E^^ + - E 



i—l i—n-\-l 
n+m 

A,: 



i— n+l 



This expression is convergent as m goes to infinity with the following limit: 



L := 



1 " 

71 ^— ^ 



A 



(n) 



where A(„) = lim,„^oo ^ X^ST+i ■'■f ^ zero - and it is not zero for most 

As - then we can choose an to so that ^ is greater than L/2. Therefore we have 
proved, that 

Proposition 3. If g < 1, then the q-expectation value of A ^ l°° , Dq p ^-^ 

{A I p''/||p''||) is not continuous if A satisfies is an n e N so that 



- + Hm — A. 



i—n+l 



^0. 



Note that a number of As satisfy this condition. 

3. Lesche stability and continuity 

The original mathematical formulation of experimental robustness by Lesche is 
not continuity, but a related notion. He introduced "normalized" values of the 
corresponding functions instead of the "bare" values in the above definition of 
continuity [HI [14]. To clarify the relation of Lesche stability and continuity we 
introduce some additional notions. Let us see then the following sets 

K {p e D I Pi = if i > n} n eN, 
V:=[jVn. 

n 

It is clear that Vm € Vn ii m < n. If p G ^ then let us define 

Up := min{n G N | p G 14,}- 

Hence Pi = if i > Up. 

Let / :V— >IR, / T^Obea function with the property 

K„ := sup{|/(p)| I p G T4} < 00 (nGN). 
It is evident, that Km < Kn, ii m < n. Moreover, there is an no G N so that 
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Definition 2: A function / with the previous properties is Lesche-stable, if 

(Ve>0)(3<5 > 0)(Vn> no)(Vr,pG K) [\\r - p\\ <S^ l/W <g 
or equivalently 

(Ve > 0){3S > 0)(Vr,p e V) ( \\r - p\\ <d^ Ifi^) ~ fip)\ < ^ 



where n := maxln^, Up} > uq. 

This definition corresponds to Lesche's original formulation [l]. 
Comparing the definitions of continuity and Lesche-stability it is clear, that 

(1) If / is uniformly continuous, then it is Lesche-stable. 

(2) If / is bounded and Lesche-stable, then it is uniformly continuous. 
Lesche stability is a global property. However, the physical meaning of experi- 
mental robustness requires a refinement which is a local property. For example let 
us see the intuitive formulation of experimental robustness of Abe |16j : 

"Given a statistical mechanical system, perform a measurement 
to obtain a probability distribution {pi}i=i....,w ■■■ ■ Perform a 
measurement again on the same system prepared in the same state 
as before. Then another probability distribution will 
be obtained." 

Continuing Abe requires that some related physical quantities do not be very 
different. 

This formulation indicates that we want that in the neighbourhood of an arbi- 
trarily given state the related physics do not change dramatically. The uniformity 
does not seem to be important. 

Therefore we introduce the following concept of stability. 

Definition 3: A function / is stable at p £V \{ 

(Ve > 0)(3(5 > 0)(Vr e V and Ur > no) ( \\r - p\\ <S^ < ^ 



A function / is stable if it is stable at all states of its domain. Lesche-stability is 
uniform stability. 

It is easy to see, that 

(1) If / is continuous in p, then it is stable there. 

(2) If / is bounded and stable in p, then it is continuous there. 

(3) If / is Lesche-stable, then it is stable everywhere. 

(4) If / is stable in a compact set of its domain, then it is Lesche-stable there. 

(5) If / is instable then it is also Lesche-instable. 

4. The stability of Renyi and Tsallis entropies and q-expectation 

VALUES 

4.1. Kq. We have shown in section \2A\ that the Renyi and Tsallis entropies are 
everywhere continuous, therefore they are stable. 

We have also seen that the q-expectation value of a physical quantity A G l°° is 
continuous everywhere, therefore the q-expectation value is stable. 

If A ^ l°°, then the q-expectation value is not necessarily continuous, however, 
it is stable. 
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In this case 



sup ■ 



P G Vn 



max l^il 



therefore, if n :— max{nr, np\ > no, then 



< 



E 



||r9|| 



The second term at the right hand side of this inequahty is the difference of the 
continuous function p — > at values p and r, as we have seen in l2.1l There- 

fore, the right hand side of this inequahty is smaller than e choosing an r closer to 
p than S = K„o/K„pe. 



4.2. q<l. In subsection 12.21 we have seen the Renyi and Tsallis entropies and the 
q-expectation values are not continuous, now we will show that they are not stable. 
We can check that by a simple modification of the proofs in 12.21 
Considering p in ((Sj and r = rg in ^ we get for the Renyi entropy, that 

K^^^y'' = log(n + m) 

and therefore the stability criteria is 

\SBir) - SnM _ log((l - 6)in^-'i + S'^m^'"') - logn^-" 



(7) 



^Renyi 



log(n + m) 



This expression converges to 1 — q as to goes to infinity. Therefore the Renyi 
entropy is instable. 

Similarly for the Tsallis entropy we get 



^Tsallis 



jl-"? - 1 



and the stability criteria is 

\ST{r) - St{p)\ 



(8) 



^Tsallis 



(1 - (1 - S)i)n^-i -59mi-« 



This expression is convergent as m goes to infinity and has the limit 



1 - ni 



Therefore choosing to so that ^ be greater than Lt/2, we see that the Tsallis 
entropy is instable. 

Regarding the stability of q-expectation values, it is enough to investigate only 
the case Ael°°. Now 

therefore the expression ^ divided by ||A||oo estimates the corresponding expres- 
sion of the stability criteria. If A has the property given in Proposition 3 then the 
q-averages are instable. 
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5. Discussion 

We have investigated some possible mathematical formulations of the experi- 
mental robustness of some physical quantities. The analysis of continuity, uniform 
continuity, and Lesche-stability revealed that these notions are closely related and 
it is convenient to introduce to use a local stability concept instead of the uniform 
notion of Lesche-stability. These formulations give essentially the same conditions 
of experimental robustness for the investigated functions: 

The Renyi and Tsallis entropies are continuous and stable if 1 < g and are not 
continuous and instable for finite uniform distributions, if g < 1. 

The q-expectation values are continuous and stable if A ^ l°° and I < q and 
are not necessarily continuous but stable if A ^ l°° and 1 < g. The q-expectation 
values are not continuous and instable for practically all physical quantities A ^ l°° 
(see the condition in l2.2p in case of finite uniform distributions. 

Observe that the proof of Lesche [Ij and Abe |i6j for Lesche instability in the 
case in the case 1 < q does not negate our stability because they do not consider 
a neighbourhood of a given distribution (e.g. formula (7) in [TBJ) but a sequence 
of finite distributions whose length goes to infinity. The proof of Abe works in the 
case q < I but it shows the instability only for a single distribution. 

If / is stable on a compact set of its domain, then it is also Lesche-stable. If / 
is instable then it is also Lesche-instable. 
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